後 Grafana 時代的第十八天 - Gafana IaC 實戰 - Dashboard、Folder - iT 邦幫忙::一起幫忙解決難題，拯救 IT 人的一天

2024 iThome 鐵人賽

DAY 18

DevOps

後 Grafana 時代的自我修養系列第 18 篇

後 Grafana 時代的第十八天 - Gafana IaC 實戰 - Dashboard、Folder

16th鐵人賽 grafana kubernetes observability grafana iac

mikehsu0618

團隊Grafana 科研遠征小隊

2024-10-02 16:28:08

178 瀏覽

分享至

前言

在前面的章節中，我們深入探討了 Grafana 中組織、團隊和用戶的 IaC 管理。現在，我們將把注意力轉向 Grafana 的核心功能之一：Dashboard 和 Folder 的功能。這些元素是 Grafana 可視化和組織管理的核心，對於創建有效的監控和分析環境至關重要。

本文將介紹如何使用 Terraform 來管理 Grafana 的 Dahboard 和 Folder，包括它們的結構、權限設置，以及如何實現靈活且可擴展的配置。

Dashboard、Folder 和其 Permission 的關係

在 Grafana 中，Dashboard 和 Folder 的關係可以簡單地用以下圖表表示：

這個結構展示了 Grafana 中 Dashboard 和 Folder 的層次關係：

根層級可以包含 Dashboards 和 Folders。
Folders 可以包含其他 Folders，形成一個層次結構。
Dashboards 可以存在於任何層級的 Folder 中，或直接位於根層級。
每個 Folder 都有自己的權限設置，可以控制對其內部資源的訪問。

這種結構提供了幾個關鍵優勢：

組織性：允許根據主題、團隊或任何其他邏輯來組織 Dashboards。
權限管理：可以在 Folder 級別設置權限，簡化了對大量 Dashboards 的訪問控制。
可擴展性：隨著監控需求的增長，可以輕鬆添加新的 Folders 和 Dashboards。
靈活性：可以根據需要調整結構，移動 Dashboards 或重組 Folders。

關於 Permission 需要注意的繼承機制

在 Grafana 中，權限可以從上層 Folder 繼承到下層 Folder 。例如，如果一個 Folder 對某個團隊授予了「View」權限，那麼該團隊的所有成員都將繼承這個權限，無需在每個 Subfolder 中重新設置權限。

這種繼承機制在管理大量 Folder 時非常有用，可以減少重複設置權限的工作量。

注意：如果某個 Folder 的權限被設定為「View」，那麼即使上層 Folder 的權限設定為「Admin」，該 Folder 仍然只能繼承到「View」的權限。

對於減少管理複雜度與功能完整的取捨

在實現 Dashboard Folder 的 IaC 時，我們需要考慮到減少管理複雜度與功能完整的取捨：

因為 Terrform 語法並不支援遞迴，所以我們不能很輕易的在 Folder 中實現 Grafana 中的無限巢狀層級的結構。必須預先設計好需要多少層級。
Folder 以及 Dashboard 資源都擁有獨立的權限管理設定，但我們利用良好的 Folder 分類設計規範，期望團隊成員日後只需針對 Folder 進行管理，而不需要針對每個 Dashboard 進行單獨的權限管理。此舉不僅減少了重複設定權限的次數，也減少了權限設定錯誤的機會，並且大幅降低我們編寫 Terraform 動態資源的複雜度。

實戰演練

讓我們深入探討如何使用 Terraform 來管理這些資源。我們將從設計全局資源結構開始，然後逐步實現各個組件。

設計全局資源結構

首先，我們定義一個靈活的結構來表示我們的 Folder 和 Dahboard：

locals {
  folders = {
    "Engineering" = {
      prevent_destroy_if_not_empty = true
      dashboards = {
        "System Performance" = {
          uid = "system-performance"
          json_file       = file("./dashboards/system_performance.json")
          additional_tags = ["performance", "system"]
        }
      }
      permissions = {
        roles = {
          "Viewer" = { permission = "Admin" }
        }
        teams = {
          "Engineering" = { permission = "Admin" }
        }
        users = {
          "alice@example.com" = { permission = "Edit" }
          "bob@example.com" = { permission = "Edit" }
        }
      }
      subfolders = {
        "Frontend" = {
          prevent_destroy_if_not_empty = true
          dashboards = {
            "Frontend Performance" = {
              uid = "frontend-performance"
              json_file       = file("./dashboards/frontend_performance.json")
              additional_tags = ["frontend", "performance"]
            }
          }
          permissions = {
            teams = {
              "Marketing" = { permission = "Edit" }
            }
            users = {
              "bob@example.com" = { permission = "Admin" }
            }
          }
        }
      }
    }
  }
  # 分離頂層 Folder 和 Subfolder
  top_level_folders = { for name, folder in local.folders : name => folder }
  sub_folders = merge([
    for parent_name, parent_folder in local.folders :
    { for sub_name, sub_folder in parent_folder.subfolders :
      "${parent_name}/${sub_name}" => merge(sub_folder, { parent = parent_name })
    }
  ]...)
  # 合併所有 Folder 資源以便後續使用
  all_folders = merge(
    grafana_folder.top_level_folders,
    grafana_folder.sub_folders
  )
}

此資料結構設計使用了嵌套的 map 結構來模擬 Grafana 中的 Folder 層次關係，使得 Folder 和 Dashboard 的管理變得簡單直觀。設計將頂層和 Subfolder 的處理邏輯分開，確保了資源創建的正確順序，同時通過 all_folders 變量的合併，為後續資源的引用提供方便性。

實現 Dahboard 資源

Dahboard 的創建也使用類似的方法：

resource "grafana_dashboard" "dashboards" {
  for_each = merge(
    // 處理頂層 Folder 的 Dashboard
    merge([
      for folder_name, folder in local.folders : {
        for dashboard_name, dashboard in lookup(folder, "dashboards", {}) :
        "${folder_name}/${dashboard_name}" => {
          folder_id = local.all_folders[folder_name].id
          json_file = dashboard.json_file
          extra_tags = lookup(dashboard, "additional_tags", [])
          uid = lookup(dashboard, "uid", null)
        }
      }
    ]...),
    // 處理 Subfolder 的 Dashboard
    merge([
      for folder_name, folder in local.folders :
      merge([
        for subfolder_name, subfolder in lookup(folder, "subfolders", {}) : {
          for dashboard_name, dashboard in lookup(subfolder, "dashboards", {}) :
          "${folder_name}/${subfolder_name}/${dashboard_name}" => {
            folder_id = local.all_folders["${folder_name}/${subfolder_name}"].id
            json_file = dashboard.json_file
            extra_tags = lookup(dashboard, "additional_tags", [])
            uid = lookup(dashboard, "uid", null)
          }
        }
      ]...)
    ]...)
  )
  folder = each.value.folder_id
  config_json = jsonencode(
    merge(
      jsondecode(each.value.json_file),
      each.value.uid != null ? { uid = each.value.uid } : {},
      {
        tags = distinct(concat(
          lookup(jsondecode(each.value.json_file), "tags", []),
          each.value.extra_tags
        ))
      }
    )
  )
}

這個 grafana_dashboard 資源有效地管理了所有層級的 Dashboard，包括頂層 Folder 和 Subfolder 中的內容。它通過嵌套循環遍歷整個 Folder 結構，為每個 Dashboard 設置必要的屬性，如 folder_id、json_file、extra_tags 和 uid。同時，它還合併了 JSON 文件中的配置與額外的標籤和 UID，實現了 Dashboard 配置富有彈性的管理。這種方法使得複雜的 Folder 和 Dashboard 結構管理變得簡單高效。

權限管理

最後，我們設置 Folder 的權限：

resource "grafana_folder_permission_item" "role_permissions" {
  for_each = merge(
    // 處理頂層 Folder 的權限
    merge([
      for folder_name, folder in local.folders : {
        for role, permission_data in lookup(folder.permissions, "roles", {}) :
        "${folder_name}/${role}" => {
          folder_uid = local.all_folders[folder_name].uid
          role       = role
          permission = permission_data.permission
        }
      }
    ]...),
    // 處理 Subfolder 的權限
    merge([
      for folder_name, folder in local.folders :
      merge([
        for subfolder_name, subfolder in lookup(folder, "subfolders", {}) : {
          for role, permission_data in lookup(subfolder.permissions, "roles", {}) :
          "${folder_name}/${subfolder_name}/${role}" => {
            folder_uid = local.all_folders["${folder_name}/${subfolder_name}"].uid
            role       = role
            permission = permission_data.permission
          }
        }
      ]...)
    ]...)
  )

  folder_uid = each.value.folder_uid
  role       = each.value.role
  permission = each.value.permission
}

// 類似的資源用於 team_permissions 和 user_permissions

這些 Permission 資源使用相同概念動態產生，包括了 Roles、Teams 和 Users 各個級別的權限管理。

實際建立資源

terraform init
terraform apply
---
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # grafana_dashboard.dashboards["Engineering/Frontend/Frontend Performance"] will be created
  + resource "grafana_dashboard" "dashboards" {
      + config_json  = jsonencode(
            // ...
        )
      + dashboard_id = (known after apply)
      + folder       = (known after apply)
      + id           = (known after apply)
      + uid          = (known after apply)
      + url          = (known after apply)
      + version      = (known after apply)
    }

  # grafana_dashboard.dashboards["Engineering/System Performance"] will be created
  + resource "grafana_dashboard" "dashboards" {
      + config_json  = jsonencode(
            // ...
        )
      + dashboard_id = (known after apply)
      + folder       = (known after apply)
      + id           = (known after apply)
      + uid          = (known after apply)
      + url          = (known after apply)
      + version      = (known after apply)
    }

  # grafana_folder.sub_folders["Engineering/Frontend"] will be created
  + resource "grafana_folder" "sub_folders" {
      + id                           = (known after apply)
      + parent_folder_uid            = "engineering"
      + prevent_destroy_if_not_empty = true
      + title                        = "Frontend"
      + uid                          = "engineering-frontend"
      + url                          = (known after apply)
    }

  # grafana_folder.top_level_folders["Engineering"] will be created
  + resource "grafana_folder" "top_level_folders" {
      + id                           = (known after apply)
      + prevent_destroy_if_not_empty = true
      + title                        = "Engineering"
      + uid                          = "engineering"
      + url                          = (known after apply)
    }

  # grafana_folder_permission_item.role_permissions["Engineering/Viewer"] will be created
  + resource "grafana_folder_permission_item" "role_permissions" {
      + folder_uid = "engineering"
      + id         = (known after apply)
      + org_id     = (known after apply)
      + permission = "Admin"
      + role       = "Viewer"
    }

  # grafana_folder_permission_item.team_permissions["Engineering/Engineering"] will be created
  + resource "grafana_folder_permission_item" "team_permissions" {
      + folder_uid = "engineering"
      + id         = (known after apply)
      + org_id     = (known after apply)
      + permission = "Admin"
      + team       = "1:40"
    }

  # grafana_folder_permission_item.team_permissions["Engineering/Frontend/Marketing"] will be created
  + resource "grafana_folder_permission_item" "team_permissions" {
      + folder_uid = "engineering-frontend"
      + id         = (known after apply)
      + org_id     = (known after apply)
      + permission = "Edit"
      + team       = "1:39"
    }

  # grafana_folder_permission_item.user_permissions["Engineering/Frontend/bob@example.com"] will be created
  + resource "grafana_folder_permission_item" "user_permissions" {
      + folder_uid = "engineering-frontend"
      + id         = (known after apply)
      + org_id     = (known after apply)
      + permission = "Admin"
      + user       = "61"
    }

  # grafana_folder_permission_item.user_permissions["Engineering/alice@example.com"] will be created
  + resource "grafana_folder_permission_item" "user_permissions" {
      + folder_uid = "engineering"
      + id         = (known after apply)
      + org_id     = (known after apply)
      + permission = "Edit"
      + user       = "62"
    }

  # grafana_folder_permission_item.user_permissions["Engineering/bob@example.com"] will be created
  + resource "grafana_folder_permission_item" "user_permissions" {
      + folder_uid = "engineering"
      + id         = (known after apply)
      + org_id     = (known after apply)
      + permission = "Edit"
      + user       = "61"
    }

Plan: 10 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 10 added, 0 changed, 0 destroyed.

接著我們可以使用 Grafana 查看我們剛剛建立的資源：

我們可以看到我們剛剛建立的 Folder 和 Dashboard 之間的階層，都如同我們在 Terraform 中設計的一樣：

我們也可以看到我們剛剛建立的權限，小細節是Subfolder 的權限會如果們預期的繼承上層 Folder 的權限：

總結

通過使用 Terraform 來管理 Grafana 的 Dashboard 和 Folder ，我們可以實現一個高度可配置、版本控制的監控環境。這種方法不僅提高了管理效率，還確保了跨環境的一致性和可重複性。隨著監控需求的增長和變化，這種基於 IaC 的方法將為您提供靈活性和可擴展性，使我們能夠輕鬆適應未來的實務需求。